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Abstract 


This  study  extends  previous  research  by  the  authors  that  focuses  on  the  growing 
global  challenge  of  affordability.  Ballooning  public  debt  burdens  are  forcing  countries 
around  the  world  to  rethink  their  approaches  to  procurement  decisions.  This  paper 
offers  a  new  approach  to  government  vendor  selection  decisions  in  major  public 
procurements.  A  key  challenge  is  for  government  purchasing  agents  to  select 
vendors  that  deliver  the  best  combination  of  desired  non-price  attributes  at  realistic 
funding  levels.  The  mechanism  proposed  in  this  paper  is  a  three-stage, 
multiattribute,  sealed-bid  procurement  auction.  It  extends  traditional  price-only 
auctions  to  one  in  which  competition  takes  place  exclusively  over  attribute  bundles. 
The  model  reveals  benefits  in  public  procurements  by  defining  an  alternative  in  terms 
of  its  value  to  the  buyer  over  a  range  of  possible  expenditures,  rather  than  as  a 
single  point  in  budget-value  space.  This  approach  leads  to  some  interesting  results. 
In  particular,  it  suggests  that  in  a  fiscally  constrained  environment,  the  traditional 
approach  of  eliminating  dominated  alternatives  could  lead  to  sub-optimal  decisions. 
The  final  extension  of  the  model  explicitly  examines  the  buyer’s  decision  problem 
under  budget  uncertainty.  The  result  is  in  a  new  metric  proposed  to  evaluate 
vendors:  an  expected  utility  measure  of  performance. 
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Introduction 

This  study  focuses  on  the  growing  global  challenge  of  affordability.  Ballooning  public 
debt  is  forcing  countries  around  the  world  to  rethink  their  procurement  strategies.  Recent 
congressional  testimony  urges  the  DoD  to  “achieve  a  balanced  mix  of  weapon  systems  that 
are  affordable”  (Written  testimony  of  M.  Sullivan  ,  2009).  In  the  absence  of  profits  to  guide 
public  procurement  decisions,  the  challenge  is  to  select  vendors  that  deliver  the  best 
possible  combination  of  desired  non-price  attributes  at  realistic  funding  levels.  The  public 
procurement  mechanism  proposed  in  this  paper  is  a  multiattribute  sealed-bid  procurement 
auction  with  multiple  budgets. 

The  U.S.  Federal  Acquisition  Regulation  (2005)  provided  guidance  in  Subpart  14.5 
on  a  two-step  procurement  process  for  government  agencies: 


Step  one  consists  of  the  request  for,  submission,  evaluation,  and  (if  necessary) 
discussion  of  a  technical  proposal.  No  pricing  is  involved.  Step  two  involves  the 
submission  of  sealed  price  bids  by  those  who  submitted  acceptable  technical 
proposals  in  step  one.  Invitations  for  bids  shall  be  issued  only  to  those  offerors 
submitting  acceptable  technical  proposals  in  step  one.  An  objective  is  to  permit 
the  development  of  a  sufficiently  descriptive  and  not  unduly  restrictive  statement 
of  the  Government’s  requirements  especially  useful  for  complex  items. 


Blondal  (2006)  discusses  a  similar  two-stage1  bidding  process,  in  which  the  procuring 
agency  issues  a  general  request  and  then  later  issues  a  detailed  request  based  on  the 
responses. 

Much  of  the  multiattribute  auction  literature,  including  Che  (1993),  Beil  and  Wein 
(2003),  and  Parkes  and  Kalagnanam  (2005),  either  implicitly  or  explicitly  includes  price 
alongside  non-price  attributes  in  the  buyer’s  value/utility  function.2  While  this  standard 
approach  is  appropriate  in  many  private-sector  contexts,  it  generates  complications  in  public 
procurements  such  as  major  defense  acquisitions.  Unlike  the  private  sector,  where  the 
incentive  to  maximize  profits  provides  a  clear  objective,  the  best  government  decision¬ 
makers  can  do  is  to  maximize  value  to  the  public  subject  to  funding  (budget)  constraints. 

In  an  application  that  maximizes  value  subject  to  a  budget  constraint,  Michael  and 
Becker  (1973)  make  the  case  that  costs  be  excluded  from  measures  of  value.  The  authors’ 
focus  is  on  performance  and  affordability.  Vendors  compete  for  a  government  contract 
based  on  their  relative  costs  of  producing  different  components  of  quality  and  their  unique 
(sunk)  technology  investments  that  define  their  ability  to  offer  different  tradeoffs  among 
these  components.  A  similar  approach  is  known  as  “cost  as  an  independent  variable” 
(CAIV).  Larsen  (2007)  offers  the  following  explanation  of  CAIV: 


1  Blondal  defined  stage  differently  than  we  do  in  this  paper.  We  use  the  term  to  refer  to  a  decision  or  set  of 
decisions  that  depends  only  on  exogenously  given  parameters  and  previous  decisions.  For  example,  Blondal 
considers  a  government  agency’s  offer  and  the  vendor  responses  to  be  a  single  stage,  whereas  we  treat  these 
as  two  distinct  stages.  Using  our  interpretation,  Blondal’s  model  is,  in  fact,  a  five-stage  process. 

2  Value  functions  are  often  referred  to  in  defense  procurement  as  measures  of  effectiveness  (MOEs).  The  term 
MOE  is  used  in  a  few  different  ways.  It  may  describe  an  attribute  itself,  a  single-attribute  value  function,  or  a 
multiattribute  value  function,  which  might  incorporate  the  whole  objective’s  hierarchy,  or  only  a  portion  of  it.  For  a 
detailed  discussion  of  MOEs,  see  Sproles  (2000).  Regardless,  this  paper  emphasizes  using  an  MOE  that 
includes  exclusively  non-price  attributes. 
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All  acquisition  programs/issues  consist  of  three  fundamental  elements:  cost, 
performance  and  schedule.  Under  CAIV,  performance  and  schedule  are 
considered  a  function  of  cost.  Cost  and  affordability  should  be  a  driving  force, 
not  an  output  after  potential  solutions  are  established,  (p.  15) 

Loerch,  Koury,  and  Maxwell  (1999)  discuss  a  Value  Added  Analysis  approach  for 
applying  multiattribute  preferences  to  optimize  the  United  States  Army’s  force  structure 
under  a  budget  constraint,  in  accordance  with  the  CAIV  concept.  The  scope  of  our  model 
differs  from  theirs,  in  that  we  focus  on  a  single  acquisition  program.  This  allows  us  to 
incorporate  vendors’  decision-making  into  the  model,  along  with  issues  of  asymmetric 
information.  In  our  model,  as  in  theirs,  prices  and  costs  do  not  appear  in  the  buyer’s  value 
function.  Instead,  the  buyer  provides  information  about  possible  budget  levels,  allowing 
prices  to  appear  in  affordability  constraints  in  the  spirit  of  CAIV. 

Budget  constraints  may  not  be  known  when  the  vendor  selection  decision  is  made. 
Buede  and  Bresnick  (1999)  describe  the  acquisition  process  as  having  four  major  phases 
and  point  out  that  vendor  selection  occurs  in  the  first  phase,  while  the  budget  may  change 
throughout  the  entire  process.  Two  pioneers  in  defense  economics,  Hitch  and  McKean 
(1967),  advocate  determining  the  maximum  effectiveness  for  a  given  budget  and  then 
examining  how  each  alternative  fares  under  several  different  budget  scenarios.  Quade 
(1989)  also  advocates  evaluating  vendor  proposals  based  on  a  range  of  possible  budgets. 
This  leads  to  the  generation  of  what  we  call  an  “expansion  path”  for  each  vendor,  which 
shows  how  the  vendor’s  proposals  change  as  the  budget  increases  or  decreases  and  thus 
provides  a  more  complete  view  of  the  vendor’s  ability  to  provide  performance.  Our  model 
allows  the  buyer  to  offer  a  set  of  possible  budget  levels  and  solicit  vendor  proposals  for  each 
one,  leading  to  the  generation  of  expansion  paths. 

Expansion  paths  reveal  valuable  information  to  government  procurement  agents. 
Suffering  from  asymmetric  information,  buyers  have  very  limited  knowledge  of  the  vendors’ 
costs  of  producing  a  particular  attribute,  as  well  as  the  technologies  (production  functions) 
that  combine  those  attributes  into  products  under  consideration.  Parkes  and  Kalagnanam 
(2005)  describe  the  vendors’  private  information:  “Seller  costs  can  be  expected  to  depend 
on  [the]  local  manufacturing  base  and  sellers  can  be  expected  to  be  well  informed  about  the 
cost  of  (upstream)  raw  materials”  (p.  437).  The  general  motivation  for  constructing  the 
expansion  paths  is  expressed  succinctly  by  Keeney  (2004):  “If  you  do  not  have  the  right 
problem,  objectives,  alternatives,  list  of  uncertainties,  and  measures  to  indicate  the  degree 
to  which  the  objectives  are  achieved,  almost  any  analysis  will  be  worthless”  (p.  200).  It  is 
imperative  in  public  procurement  for  alternatives  to  be  adequately  described  and  for  any 
budget  uncertainty  to  be  explicitly  acknowledged.  We  emphasize  that  this  can  be  carried 
out  using  a  value-focused  thinking  approach,  as  discussed  by  Keeney  (1992)  and  by  Parnell 
(2007)  in  the  context  of  national  defense.  That  is,  it  is  important  for  the  buyer’s  evaluation 
process  to  be  carried  out  independent  of  the  particular  alternatives  offered. 

In  the  Model  section,  we  introduce  our  proposal  for  a  three-stage  procurement 
model.  This  multiattribute  sealed-bid  procurement  auction  emphasizes  the  use  of  a  value 
function  with  exclusively  non-price  attributes  and  the  specification  of  a  set  of  possible  budget 
levels.  We  formulate  the  decision  problems  faced  by  the  buyer  and  the  vendors,  and 
discuss  various  insights  derived  from  the  model.  We  also  provide  two  historical  examples  of 
government  procurement  decisions  that  likely  could  have  benefited  from  a  more  complete 
formulation  of  alternatives  and  specification  of  uncertainties. 
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After  vendor  bids  have  been  solicited  for  a  spectrum  of  possible  budget  levels,  the 
Budget  Uncertainty  section  expands  the  formulation  of  the  buyer’s  problem  to  explicitly 
include  the  buyer’s  beliefs  of  the  probability  associated  with  various  budget  levels.  We 
follow  a  decision  under  uncertainty  approach  as  introduced  by  Pratt,  Raiffa,  and  Schlaifer 
(1964).  In  addition  to  expressing  their  beliefs  about  various  budget  levels  as  probabilities, 
the  government  buyer  specifies  a  utility  function  over  the  value  of  attribute  bundles  that 
incorporates  his  or  her  risk  attitude,  as  discussed  by  Dyer  and  Sarin  (1982)  and  Matheson 
and  Abbas  (2005).  The  result  is  a  new  metric  proposed  to  evaluate  vendors:  an  expected 
utility  measure  of  performance. 

Model 


The  procurement  agency  (the  buyer)  begins  by  specifying  a  multiattribute  value 
function  over  a  set  of  desired  attributes  A  =  {a1(  ...,  an},  as  well  as  a  set  of  (increasing) 
possible  budget  levels  B  =  {bu  ...,  bk}.  There  are  m  vendors,  each  of  whom  will  respond  in 
the  second  stage  with  a  bid.  A  bid  consists  of  a  set  of  attribute  levels  that  can  be  produced 
by  a  vendor  for  each  of  the  k  possible  budget  levels.  Vendor / s  bid  can  be  expressed  as  k 

vectors  of  the  form  A/  =(a1;.,...,aJ?/)  for  j  =  1  ,  where  afj  is  the  level  of  attribute  / 

offered  by  vendor  j.  Note  that  unlike  bids  in  most  multiattribute  auctions,  Aj  does  not  include 
any  information  about  price.  Instead,  the  price  is  captured  in  the  multiple  possible  budget 
constraints.  The  buyer’s  ultimate  decision  (the  third  stage)  is  to  select  a  vendor 
j  e  {l, .  The  buyer’s  preferences  over  the  attributes  are  represented  by  a  value 

function  ■  The  same  value  function  is  used  for  all  possible  realized  budget  levels. 

For  ease  of  exposition,  we  assume  V  [Aj  j  is  an  additive  multiattribute  value  function 
similar  to  that  discussed  by  Keeney  and  Raiffa  (1976)  and  Kirkwood  (1997),  although  it  is 
later  demonstrated  the  conclusions  of  the  paper  do  not  require  V  ( A ; )  to  be  additive.  The 

use  of  additive  multiattribute  value  functions  requires  the  assumption  of  mutual  preferential 
independence  (Dyer  &  Sarin,  1979;  Kirkwood  &  Sarin,  1980).  This  implies  that  alternatives 
can  be  compared  exclusively  on  the  set  of  attributes  over  which  they  differ,  ignoring  levels  of 
other  attributes. 

For  any  given  budget  level,  the  buyer’s  objective  is  as  follows: 

max  FG)  =  I>,v,(a  ) 

'  M  (D 

where  wt  is  the  weight  the  buyer  places  on  attribute  /':  0 <wt  <1,  and  =1,  and  v,  [a A  is 

the  buyer’s  single-attribute  value  function  for  attribute  /.  We  assume  that  i/,(a,j)  is  scaled 
such  that  the  minimum  achievable  value  is  zero  and  the  maximum  achievable  value  is  one. 

Note  that  since  V  ( A  j )  is  a  weighted  average  of  terms  between  zero  and  one,  it  also  ranges 

from  zero  to  one.  We  assume  the  buyer  has  an  understanding  of  the  range  of  attribute 
levels  in  determining  the  weights  and  that  the  buyer  explicitly  shares  the  weights  and  the 
single-attribute  value  functions.  It  is  necessary  for  the  government  buyer  to  completely 
specify  its  preferences  to  the  vendors  by  providing  w ,  and  \/((a,y)  for  /  =  1,  ...,  n.  The  final 
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stage  of  the  model  involves  applying  Equation  1  to  the  set  of  vendor  bids  and  the  buyer 
selecting  the  vendor  that  yields  the  highest  value. 

Given  the  buyer-determined  set  of  desired  attributes  A,  along  with  the  weights  and 
single-attribute  value  functions,  and  the  set  of  possible  budget  levels  B,  each  vendor 
produces  an  attribute  bundle  to  submit  to  the  buyer  for  each  of  the  k  possible  budget 
constraints.  Since  vendors  have  private  information  about  their  own  production  capabilities, 
costs,  and  profit  requirements,  each  vendor  forms  his  or  her  own  private  beliefs  about  the 
likelihood  of  a  bid  being  accepted.3  We  assume  that  all  vendors  believe  the  probability  of  a 

bid  being  accepted  is  increasing  in  U  ( /E )  for  all  possible  budget  levels. 

The  problem  faced  by  a  representative  vendor  j  for  an  arbitrary  budget  level  b  can  be 
expressed  as  follows: 


max  V(Aj)  =  iwvvi{av),  i  = 

aij  1=1 

subject  to  CJ(vi(alj),...,vn(anj))<b,  (2) 

where  Cj  is  the  total  cost  paid  by  firm  j  (with  the  desired  profit  margin  included)  to  produce  a 
set  of  single-attribute  values.  The  cost  incurred  to  generate  the  corresponding  attribute 
bundle  cannot  exceed  b.  We  assume  that  C,  is  increasing  in  vt  for  all  /  and  that  Cy  is  strictly 

convex.  This  condition  is  not  overly  restrictive,  since  it  simply  implies  decreasing  returns 
from  vendor  investments  to  improve  any  individual  attribute  value.  Because  the  objective 
function  in  Equation  2  is  linear,  given  the  assumed  properties  of  a  representative  vendor’s 
cost  function,  a  unique  solution  (vendor  proposal)  will  exist. 

For  purposes  of  illustration,  and  ease  of  exposition,  the  remainder  of  this  study 
focuses  on  two  vendors  and  two  (non-price)  attributes.  The  two  vendors  can  have  different 
technologies  with  which  to  combine  the  two  attributes  and  may  face  different  costs  to 
improve  individual  attributes.  The  Lagrangian  function  to  solve  the  vendor’s  problem  is 
given  by  the  following: 

Lj  =  w, v,  (a,, )  +  vv2v2  (a2J ) - /l,  (& - CJ  (v,  (atJ),v2  (a2/ )))  fory'=  1,2.  (3) 

Since  an  improvement  in  either  attribute  increases  the  value  of  a  particular  attribute  bundle 
to  the  buyer,  or  >  o,  each  vendor  will  use  the  total  available  budget  b  to  produce  its 

attribute  bundle  proposal.  In  this  case,  first  order  necessary  conditions  for  an  optimum  are 
given  by  the  following: 


3  For  simplicity,  we  assume  that  each  vendor  determines  its  required  profit  margin  for  each  possible  budget  level 
proposed  by  the  buyer  and  that  these  fixed  profit  margins  are  incorporated  into  the  attribute  bundles  offered.  We 
focus  on  the  vendor’s  decision  of  how  to  allocate  fixed  amounts  of  funding  across  the  set  of  attributes  to 
maximize  the  value  provided  to  the  buyer.  Although  our  results  do  not  require  any  more  details  of  vendor 
behavior ,  we  believe  this  would  be  an  interesting  avenue  for  future  research.  This  exploration  could  be  based 
on  a  vendor’s  search  for  an  optimum  bidding  strategy  in  a  Dutch  auction  (see  McAfee  &  McMillan,  1987,  or 
Milgrom,  1989),  which  requires  a  complete  formulation  of  the  bidder’s  beliefs,  values,  and  risk  attitude. 
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dL.  dCi 

-=W,  +  A;  - -  =  0 


dv 


dv j 


dL.  dC, 

L=Wn  +  A:  - -  =  0 


dv. 


dv , 


dJd 

dA. 


■h  —  Cj  (V1  (ai7))V2  (a2y  ))  : 


where  Equation4c  simply  asserts  that  the  total  budget  is  being  used, 
and  4b  yields 


(4a) 

(4b) 

0 ,  (4c) 

Solving  Equations  4a 


(5) 


This  implies  that  the  optimum  strategy  for  each  vendor  is  to  choose  a  bid  that  uses 
the  entire  budget  and  for  which  the  two  attributes  have  equal  ratios  between  the  weight 
placed  on  the  attribute  by  the  buyer  and  the  vendor’s  marginal  cost  of  increasing  the  value 
provided  by  that  attribute.4  With  two  competing  vendors,  there  will  be  two  bids  that  can  be 

represented  by  attribute  bundles:  (an,a21)  and  (a12,a22). 


Of  course,  cost  functions  are  likely  to  vary  across  vendors,  meaning  that  the 
marginal  costs  in  Equation  5  are  likely  to  vary  across  vendors  as  well,  resulting  in  a 
potentially  diverse  set  of  bids.  Multiattribute  auctions  allow  vendors  to  differentiate 
themselves  in  the  auction  process  and  to  bid  on  their  competitive  advantages  (Wise  and 
Morrison,  2000). 

With  the  buyer’s  preferences  and  the  vendor’s  bidding  strategy  in  place,  we  now 
demonstrate  how  a  buyer  can  explore  important  differences  between  vendors.  Each  vendor 
goes  through  the  process  described  above  for  the  k  different  budget  estimates,  each  time 
producing  a  bid  that  satisfies  Equation  5  for  each  of  the  k  possible  budgets.  This  set  of  bids 
from  a  vendor  constitutes  an  expansion  path.  It  tells  the  buyer  precisely  how  a  vendor’s  bid 
will  change  as  the  budget  constraint  is  relaxed  (or  tightened).  For  purposes  of  illustration, 
throughout  the  remainder  of  the  paper,  we  use  a  set  of  six  possible  budget  levels  to  simulate 
alternative  possible  funding  constraints:  ($5M,  $10M,  $15M,  $20M,  $25M,  $30M)  or  simply 
(5,10,15,20,25,30). 

Consider  the  following  functional  form  for  the  cost  functions: 


C,(v\ayi),v2(a2j)) 


aUe 


P\j* 


+  a2Je 


Pi  jvi(a: 


ij) 


aXj  ,a2J,  J3Xj  ,  p2j  >  0  for  j  =  1, 2 


(6) 


This  particular  functional  form  is  separable,  in  that  it  consists  of  the  sum  of  cost 
functions  on  the  individual  attributes.  Each  individual  attribute  cost  function  is  increasing 
and  convex,  where  the  exponent  /T  in  Equation  6  determines  the  convexity  of  each 


4  Note  that  Equation  5  has  a  unique  solution  for  each  vendor  when  the  entire  budget  is  being  used.  Because  the 
cost  function  is  strictly  convex,  as  we  move  along  the  budget  constraint  curve,  the  marginal  cost  of  improving  one 
attribute’s  value  is  increasing,  and  the  marginal  cost  of  improving  the  other  attribute’s  value  is  decreasing. 
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function.  Although  the  results  of  the  study  do  not  depend  on  this  particular  functional  form, 
this  offers  a  relatively  simple  way  to  illustrate  our  expansion  path  approach  to  government 
vendor  selection  decisions. 

Figure  1  offers  an  example  of  an  expansion  path.  The  buyer  in  this  example  places 
a  weight  of  0.7  on  Attribute  1  and  0.3  on  Attribute  2.  The  vendor  represented  in  Figure  1 , 
whom  we  will  refer  to  as  Vendor  1,  faces  lower  marginal  costs  to  improve  Attribute  1  than  to 
improve  Attribute  2  at  low  levels.  Specifically, 

an  =  2.2, a21  =  2.1,  =  2.0, /?21  =  1.7  .  (7) 


Figure  1.  Expansion  Path 

Note.  This  graph  shows  the  expansion  path  for  a  vendor  as  the  budget  increases  from  5  to 
30.  The  markers  of  increasing  size  show  the  vendor’s  attribute  bundle  proposals  as  the 
budget  increases  in  increments  of  5. 

Expansion  paths  will  differ  among  vendors  if  the  parameters  of  their  cost  functions 
( , (ij )  differ.  Consider  a  second  vendor  (Vendor  2),  whose  individual-attribute  cost 

functions  are  more  convex.  Specifically, 

an  =1.5 ,a22  =1.5 ,/?12  =  2.7, /?22  =  2.7  .  (8) 

Vendor  2  is  symmetric  in  the  sense  that  he  or  she  does  not  specialize  in  providing  a 
particular  attribute.  Any  asymmetry  in  Vendor  2’s  expansion  path  is  due  to  the  buyer  having 
asymmetric  preferences  over  the  two  attributes. 

Applying  the  parameters  in  Equations  7  and  8  results  in  the  expansion  paths  shown 
in  Figure  2.  The  two  piecewise  linear  expansion  paths,  one  for  each  vendor,  are  based  on 
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the  six  possible  budget  levels.5  They  illustrate  optimum  combinations  of  attribute  values  that 
can  be  produced  by  each  vendor  and  offered  to  the  buyer  at  the  different  funding  levels. 


Expansion  Paths  -  Differing  Cost 
Functions 


Vi 


Figure  2.  Expansion  Paths — Differing  Cost  Functions 

Note.  This  graph  shows  the  expansion  paths  for  two  vendors  with  differing  cost  functions  as 
the  budget  increases  from  5  to  30.  The  markers  of  increasing  size  show  each  vendor’s 
attribute  bundle  proposals  as  the  budget  increases  in  increments  of  5. 

Figure  2  reveals  an  interesting  dynamic,  which  relates  to  one  of  the  key  insights  of 
this  study.  Under  optimistic  assumptions  about  future  budgets,  it  is  clear  that  Vendor  1  will 
be  preferred  and  selected  as  the  winner.  At  relatively  high  budgets,  Vendor  1  dominates 
Vendor  2.  However,  the  reverse  is  true  under  a  more  pessimistic  budget.  Under  severe 
budget  constraints  (e.g.,  $5  million),  it  is  clear  that  Vendor  2  will  be  preferred  and  selected 
as  the  winner.  If  a  government  buyer  believes  a  significant  budget  cut  is  possible  then 
selecting  a  dominant  alternative  under  the  optimistic  budget  scenario  (Vendor  1 )  may  be 
misleading.  The  dominated  alternative  (Vendor  2)  should  not  be  prematurely  eliminated 
since  it  may,  in  fact,  end  up  being  the  preferred  vendor. 


5  Fitting  a  curve  to  the  points  might  also  be  a  reasonable  approach.  We  use  a  piecewise  linear  form  because  we 
specifically  would  like  every  attribute  bundle  in  the  vendor’s  bid  to  fall  on  the  expansion  path  because  we  believe 
this  makes  the  method  more  transparent.  We  would  advise  the  analyst  and  the  buyer  to  use  their  discretion  on 
which  approach  to  take,  based  on  the  particular  context  of  the  auction. 
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To  illustrate  this  new  expansion  path  approach  more  clearly,  we  compute  v(Aj)  for 

each  of  the  twelve  attribute  pairs  shown  in  Figure  2.  The  two  vendors’  bids  can  then  be 
plotted  as  curves  in  “budget-value”  (or  cost-effectiveness)  space,  as  shown  in  Figure  3. 

Value  by  Budget  Level 


Figure  3.  Value  by  Budget  Level 

Note.  This  graph  shows  the  value  provided  by  each  vendor’s  bid  for  various  budget  levels. 

Related  to  the  expansion  paths,  the  bids  illustrated  in  Figure  3  are  piecewise  linear 
curves.  We  can  think  of  each  one  as  a  function  expressing  the  value  to  the  buyer  of  the 
attribute  bundles  each  vendor  will  provide  over  the  range  of  possible  budget  levels.  We  will 
write  this  function  for  vendor  j  as  Q/b),  defined  for  all  possible  budget  levels  b. 

The  dynamic  revealed  in  Figure  2  is  illustrated  more  clearly  in  Figure  3.  It  is 
apparent  from  Figure  3  that  Vendor  2  dominates  the  competition  for  any  positive  budget 
below  the  switch-point,  b  <  b\  while  Vendor  1  dominates  for  any  budget  above  the  switch- 
point,  b  >  b’.  As  Quade  (1989)  also  discusses,  this  observation  suggests  rethinking  the 
simpler  definition  of  dominance,  which  refers  to  points  (not  functions)  in  cost-effectiveness 
space. 

Viewing  alternatives  as  functions  in  budget-value  space  reveals  that  the  point-based 
definition  can  be  misleading.  A  static  comparison  that  begins  by  assuming  a  relatively  high 
fixed  budget  would  eliminate  Vendor  2  from  further  consideration.  For  example,  consider 
offers  from  Vendor  1  and  Vendor  2  based  on  optimistic  budgets  above  b’.  A  technique  that 
focuses  on  points  and  not  functions  would  eliminate  Vendor  2;  yet,  Figure  3  indicates  that 
eliminating  Vendor  2  prematurely  could  lead  to  a  less  desirable  outcome  if  subsequent 
budget  cuts  resulted  in  an  actual  budget  somewhere  in  the  range  of  0  <  b  <  b’.  This 
observation  suggests  the  need  for  a  new  approach  to  government  vendor  selection 
decisions. 
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This  switch-point  phenomenon  occurs  as  a  result  of  differences  in  the  two  vendors’ 
expansion  paths.  There  is  nothing  unique  about  the  particular  functions  chosen  in  our 
example.  The  same  results  can  be  obtained  in  many  different  ways,  including  with  non¬ 
additive  forms  of  the  buyer’s  value  function.  In  fact,  non-linear  interactions  between 
attributes  are  likely  to  magnify  this  effect.6 

While  the  approach  in  this  paper  involves  assessing  the  expansion  paths  by  soliciting 
vendors’  attribute  bundle  offers  for  multiple  budgets,  it  may  be  possible  for  a  government 
buyer  to  obtain  similar  information  by  soliciting  price  bids  for  multiple  sets  of  performance 
requirements  (i.e. ,  specified  attribute  levels).  This  would  have  the  advantage  of  not 
requiring  the  buyer  to  reveal  a  value  function,  but  also  the  corresponding  disadvantage  of 
not  allowing  each  vendor  the  flexibility  to  achieve  the  desired  values  with  the  least  costly 
combinations  of  attribute  levels.  Using  either  approach,  the  buyer  benefits  by  being  able  to 
incorporate  affordability  into  the  decision  in  a  meaningful  way  when  the  budget  is  not  known 
with  certainty.  In  particular,  the  buyer  gains  the  ability  to  view  each  alternative  as  a  function 
in  cost-effectiveness  space,  rather  than  as  a  single  point. 

Selecting  a  vendor  based  on  points  in  cost-effectiveness  space  can  lead  to  worse 
outcomes  than  expected,  since  there  may  be  uncertainties  present  that  are  implicitly 
ignored.  One  example  is  the  $8.8  billion  U.S.  Navy  and  Marine  Corps  Intranet  (NMCI) 
contract,  which  was  awarded  to  Electronic  Data  System  (EDS)  in  2000.  Wilson  (2006) 
explains  that  EDS  was  the  lowest  bidder  and  that  problems  arose  due  to  the  scope  of  EDS’ 
task  being  much  larger  than  expected  by  either  party.  Whether  another  vendor  might  have 
performed  better  than  EDS  given  the  expanded  scope  is  unknown.  (See  Jordan,  2007,  for 
more  information  on  NMCI.) 

A  second  example  is  the  U.S.  Air  Force’s  acquisition  of  the  Boeing  (then  McDonnell 
Douglas)  C-17  Globemaster  III.  This  aircraft,  commonly  referred  to  as  the  C-17,  is  used  as 
an  airlifter  for  troops  and  cargo.  McDonnell  Douglas’  C-17  proposal  was  selected  in  1981, 
effectively  ending  the  bidding  process.  However,  a  dollar  amount  was  not  specified  until 
1986,  when  the  Air  Force  awarded  McDonnell  Douglas  a  $3.39  billion  contract.  Even  after 
1986,  the  C-17  program  was  subjected  to  a  great  deal  of  change.  Kennedy  (1999)  explains 
the  following: 

In  addition,  how  much  airlift  was  required  for  war  plans  was  largely  undefined. 
Securing  necessary  funding  for  the  C-17  was  simply  an  ordeal.  That  the 
program’s  funding  fell  victim  to  the  budget  axes  wielded  by  Congress,  DoD,  and 
Air  Force  undermined  the  ultimate  goal — timely  operational  delivery  of  the  C-17. 

As  in  the  NMCI  example,  it  would  have  been  very  difficult  to  foresee  the  eventual  outcome 
for  the  C-17  based  simply  on  a  cost-effectiveness  point  when  the  decision  was  made. 

The  sensitivity  of  vendor  selection  decisions  to  different  funding  scenarios  is  a 
fundamental  result  that  arises  in  a  wide  variety  of  government  procurement  contexts  and 
places  a  premium  on  affordability.  In  a  constrained  fiscal  environment,  we  strongly 
recommend  the  adoption  of  an  expansion  path  approach  to  guide  government  vendor 
selection  decisions. 


6  For  example,  consider  a  multiplicative  value  function  and  suppose  that  one  vendor  has  to  incur  a  large  cost  to 
increase  the  value  from  0  to  0.1  for  one  particular  attribute.  This  vendor  will  offer  bids  of  little  value  for  low 
budgets  but,  depending  on  cost  functions,  may  offer  very  attractive  bids  for  higher  budgets. 
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Budget  Uncertainty 

A  natural  extension  of  the  model  is  to  consider  a  procurement  auction  in  which  the 
buyer  assigns  a  probability  distribution  over  the  set  of  possible  budgets.  If  the  buyer 
believes  that  the  realized  budget  will  be  b  with  probability  p(b)  or,  in  the  continuous  case, 
that  b  has  a  probability  density  function  f(b)  then  the  government  vendor  selection  problem 
can  be  examined  using  a  decision  under  uncertainty  approach. 

This  adds  a  valuable  new  layer  to  the  problem:  We  must  now  include  the  buyer’s  risk 
attitude,  because  he  or  she  will  be  evaluating  gambles  over  multiple  possible  values.  We 
express  risk  attitudes  through  a  utility  function  U,  which  takes  the  overall  multiattribute  value 
measure  as  its  argument  (see  Dyer  &  Sarin,  1982,  or  Matheson  &  Abbas,  2005,  for  details). 
This  approach  allows  us  to  separate  the  buyer’s  attitude  toward  risk  and  their  strength  of 
preferences  over  the  attributes. 

Given  a  value  function  V  and  maximum  and  minimum  achievable  values,  U  can  be 
assessed  using  simple  binary  gambles.  For  example,  the  buyer  could  specify  an  attribute 
bundle  a0  that  provides  the  minimum  value  (zero)  and  an  attribute  bundle  a*  that  provides 
the  maximum  value  (one)  and  then  consider  a  hypothetical  gamble  in  which  he  or  she 
receives  a*  with  probability  p  and  a0  with  probability  1-p.  For  any  other  attribute  bundle  a’, 
U(V(a’))  would  simply  be  the  value  of  p  for  which  the  buyer  is  indifferent  between  receiving 
the  uncertain  gamble  and  a  certain  value,  a’. 

The  government  buyer’s  new  problem  is  to  select  a  vendor  j  to  maximize 

2>(*  MrM).  <9> 

b 

or,  in  the  continuous  case,  to  maximize 

\f{b)u{eij{b))db.  (io) 

That  is,  the  government  buyer  maximizes  the  expected  utility  provided  by  the  vendor, 
incorporating  both  the  strength  of  its  preferences  over  the  vendor’s  attribute  bundle 
proposals,  expressed  by  Q;,  and  its  risk  attitude,  expressed  by  U. 

Consider  both  the  buyer  and  vendors’  information  used  to  generate  Figure  2.  Recall 
that  the  buyer  places  weights  of  0.7  and  0.3  on  Attributes  1  and  2,  respectively,  while 
individual  vendor  production  and  cost  characteristics  are  given  by  the  parameters  in 
Equations  7  and  8).  Now  suppose  the  buyer  has  the  exponential  utility  function7 

1  _  ~2V 

u(v)=f^-  CD 

where,  as  previously  specified,  V  varies  between  zero  and  one  over  the  possible  attribute 
bundles.  The  function  and  parameters  given  by  Equation  1 1  represent  a  decision-maker 
who  is  risk  averse.  Note  that  since  the  minimum  value  of  V  is  zero  and  the  maximum  is  one, 
U(V)  also  varies  between  zero  and  one.  Figure  4  illustrates  the  values  and  corresponding 


7  We  chose  the  exponential  function  because  it  has  constant  absolute  risk  aversion,  measured  by  a  risk 
tolerance  parameter  (in  this  case,  0.5),  making  its  assessment  reasonably  straightforward  and  understandable. 
It  is  commonly  used  in  decisions  under  uncertainty,  but  the  analysis  could  certainly  be  carried  out  using  a 
different  class  of  utility  function  if  desired. 
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utilities  to  the  buyer  of  each  vendor’s  attribute  bundle  proposals  under  the  six  possible 
budget  scenarios,  superimposed  on  the  utility  function  defined  by  Equation  11. 

Bid  Utilities 


Figure  4.  Bid  Utilities 

Note.  This  graph  shows  the  buyer’s  utility  function  and  the  value  and  corresponding  utility 
offered  by  each  vendor  for  the  six  budget  scenarios  in  the  decision  under  uncertainty 
example. 

Consider  a  scenario  in  which  the  buyer  believes  that  51;  ...,  be  will  occur  with 
probabilities  0.1,  0.15,  0.35,  0.25,  0.1,  and  0.05,  respectively.  Given  these  probabilities  for 
the  six  budget  levels  and  this  particular  buyer’s  preferences,  the  expected  utility  if  Vendor  1 
is  selected  is  0.771,  as  opposed  to  0.800  if  Vendor  2  is  selected.  While  this  aggregate 
result  suggests  that  our  buyer  should  select  Vendor  2,  disaggregating  the  vendor  selection 
problem  offers  additional  insights. 

The  bundle  of  attributes  provided  by  Vendor  1  would  be  more  desirable  for  budget 
levels  1 5,  20,  25,  and  30,  one  of  which  is  likely  to  occur  with  a  probability  of  0.75.  However, 
in  the  case  of  a  very  low  budget,  Vendor  1’s  attribute  bundle  would  be  far  less  desirable. 
Yet,  the  expected  values  of  the  two  bids  are  nearly  identical.  Such  insights  would  be  nearly 
impossible  to  obtain  when  presented  with  only  a  single  bid  from  each  vendor  for  the  most 
likely  budget,  5=15.  More  revealing  and  robust  analysis  is  only  feasible  if  the  buyer  solicits 
bids  from  the  vendors  over  multiple  possible  budget  levels. 

Constructing  a  gamble  over  possible  overall  values  is  extremely  difficult  if  a  vendor’s 
bid  consists  of  only  one  attribute  bundle  for  a  single  budget,  rather  than  a  set  of  attribute 
bundles  for  multiple  budgets.  A  decision  under  uncertainty  approach  requires  decision¬ 
makers  to  place  a  value  on  all  possible  outcomes.  The  procurement  auction  framework 
advocated  in  this  paper  ensures  that  these  outcomes  are  fully  specified. 
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Conclusion 


This  paper  offers  a  new  approach  to  government  vendor  selection  decisions  in  major 
public  procurements.  The  paper  describes  a  simple  three-stage,  multiattribute  procurement 
process  for  government  vendor  selection  decisions.  It  allows  the  buyer  to  incorporate  the 
government’s  preferences  over  multiple  attributes,  and  it  allows  each  vendor  to  offer  its  best 
possible  bid  based  on  the  budget  estimate  for  the  program  and  on  each  vendor’s  cost 
structure.  The  model  operationalizes  a  version  of  the  popular  concept  of  cost  as  an 
independent  variable  (CAIV).  The  results  of  this  study  reveal  the  importance  in  the  public 
sector  of  including  costs  as  part  of  a  budget  constraint,  rather  than  incorporating  costs 
directly  in  the  buyer’s  value  and  utility  function. 

The  model  developed  in  this  paper  allows  vendors  to  submit  bids  for  a  range  of 
possible  budget  levels.  This  leads  to  the  generation  of  an  expansion  path  for  each  vendor, 
which  illustrates  how  each  vendor’s  bid  improves  as  budgets  increase.  Most  importantly,  it 
is  demonstrated  that  a  vendor  whose  bid  is  dominated  at  one  particular  budget  level  can 
easily  end  up  being  the  winner  at  another  budget  level.  This  makes  it  vital  for  procurement 
agencies  to  rethink  traditional  public  sector  bid  solicitations.  Instead  of  viewing  each  vendor 
as  a  single  point  in  cost-effectiveness  space,  it  is  important  for  governments  to  view  each 
vendor  as  a  curve  in  budget-value  space.  In  economies  where  affordability  is  a  priority  and 
where  budgets  are  likely  to  change  over  time,  the  approach  proposed  in  this  paper  can 
result  in  better  choices  for  voters  and  taxpayers  since  it  ensures  vendors  are  not 
prematurely  eliminated  from  consideration. 

Finally,  since  precise  funding  levels  may  not  be  known  with  certainty  when  vendor 
selection  decisions  are  made,  we  explicitly  model  vendor  selection  as  a  decision  under 
uncertainty.  In  this  case,  the  buyer  assigns  a  probability  distribution  over  all  possible 
budgets  (funding  levels)  while  a  utility  function  captures  the  buyer’s  attitude  toward  risk. 

This  methodology  enables  buyers  to  generate  expected  utilities  from  vendor  proposals, 
providing  a  valuable  new  approach  and  metric  for  government  vendor  selection  decisions. 

The  approach  in  this  paper  can  be  thought  of  as  a  strategic  choice  of  auction 
mechanism  for  a  buyer  when  a  range  of  budget  authorities  for  the  program  can  be  estimated 
and  products  are  differentiated  and  complex.  The  approach  combines  the  competitive 
advantages  of  auctions  with  the  flexibility  of  decisions  based  on  multiple  attributes  of  a 
product,  all  while  incorporating  considerations  of  affordability  when  the  budget  level  is  not 
known  with  certainty. 
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Research  Program: 
Creating  Synergy  for  Informed 


Military  Cost-Benefit  Analysis: 
Introducing  Affordability  in  Vendor  Selection 


Francois  Melese,  Jay  Simon,  Anke  Richter 

Defense  Resources  Management  Institute  (DRMI) 
Naval  Postgraduate  School 


Background 

•  Ballooning  public  debt  forces  DoD  to 
rethink  its  procurement  strategy. 


•  Congressional  testimony  urges  DoD 
to  “achieve  a  balanced  mix  of  weapon 
systems  that  are  affordable ” 


(M.  Sullivan,  GAO  2009) 
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Background 


Cost  as  an  Independent  Variable 

(CAIV) 


•  “Cost  and  affordability  should  be  a 
driving  force  not  an  output  after 
potential  solutions  are  established.” 

(Larsen,  2007  p.  15) 
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Background 

•  Hitch  and  McKean  (1967),  advocate 
determining  the  maximum  effectiveness 
for  a  given  budget,  and  then  examining 
how  each  alternative  fares  under  several 
different  budget  scenarios. 


•  Quade  (1989)  advocates  evaluating 
vendor  proposals  based  on  a  range  of 
possible  budgets. 
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Introduction 

•  Procurement  Goal: 

-Select  vendors  that  deliver  the  best 
combination  of  desired  non-price 
attributes  at  realistic  funding  levels. 


•  New  approach  to  vendor  selection: 


-Multi-attribute  sealed-bid  procurement 
auction  with  multiple  budgets. 


Acquisition  Research  Program:  Creating  Synergy  for  Informed  Change 


Monlcrej,  W 

5 

Introduction 


•  Goals: 

-  Provide  a  more  complete  view  of  vendor’s  ability  to  perform 
under  different  budget  scenarios. 

-  Develop  a  new  Vendor  Selection  Metric  (VSM)  for  vendor 
selection  decisions 

•  Three  Stage  Procurement  Model 

1) Government  offers  a  set  of  possible  funding  levels. 

2) Vendors  offer  proposals  for  each  budget. 

3) Government  selects  vendor. 


•  “Expansion  paths”  for  each  vendor,  reveal  how 
vendor  proposals  change  as  funding  changes. 
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Three  Stage  Procurement  Model 

•  Based  on  “Economic  Evaluation  of  Alternatives”  (EEoA)* 

1)  DoD  reveals  desired  attributes  and  a  set  of 
possible  funding  levels  for  the  program 

2)  Vendor  proposals  consist  of  sets  of  non¬ 
price  performance  attributes  for  each 
possible  funding  level 


3)  DoD  selects  vendors  according  to  its 
weighting  of  attributes  (i.e.  a  multi-attribute 
value/utility  function) 


*  Page  25-28  in  Melese,  F.  “The  Economic  Evaluation  of  Alternatives,”  Proceedings  of  the  6th  Annual 
Acquisition  Research  Symposium:  Defense  Acquisition  in  Transition,  Vol  1 . _ 
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Model 


•  n  vendors 

•  m  attributes  (A) 

•  k possible  budget  levels,  B  (1,...,/c) 

•  Vendor  offers  4  =  [an,...,aim]  for  each  funding  level 

•  DoD  value  function  (MOE)  is:  v(Ai) 

•  For  each  budget  level,  b,  DoD’s  objective  is: 


Vendor’s  Decision  Problem 


•  For  each  possible  budget  level,  b,  Vendor  i’s 
problem  can  be  expressed  as  offering  a  mix 
of  attributes  that: 

m  , 

max  V(Ai)=£u’/v/(ai/ 

a«  7=1 


TYl 

S.t.  TC,  =  2c„(<i„)<4 

7=1 
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Simplified  Example 


•  For  simplicity,  analysis  assumes: 

Two  attributes 
Two  vendors 


Vendors  can  differ  in  their  cost  functions 
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Price  &  Performance  Bid 
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Individual  Vendor  Offers  over  a  Range  of  Budgets 


Value  (MOE)  for  each  Budget 
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Budget  Uncertainty 

Challenges: 

-Optimal  vendor  choice  can  change  with 
changes  in  the  budget 

-Large  &  rising  federal  debt  results  in 
shrinking  discretionary  defense  budget 

=>  Increasing  Budget  Uncertainty 
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Budget  Uncertainty 


•  If  we  can  assign  probabilities  to 
the  possible  budget  levels,  we  can 
use  expected  utility  as  a  vendor 
selection  metric  for  the  economic 
evaluation  of  alternatives 
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Vendor  Selection  Metric  (VSM) 


•  Vendor  Selection  Metric  (VSM)  is  an 
expected  utility  function  that  depends 
on: 


a) the  decision  maker’s  beliefs  of  the 
likelihood  of  each  budget  level 

b) the  relative  preferences  of  the 
attributes  offered,  and 


c)  attitude  toward  risk 
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Value 


Suppose  DoD  believes  these  are  the  probabilities 
associated  with  each  funding  level 


Value  (MOE)  for  each  Budget 
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Vendor  Selection  Metric  (VSM): 

•  Given  these  probabilities  for  the  six  budget 
levels  and  assumptions  about  DoD’s  value 
function  and  risk  aversion,  the  vendor  selection 
metric  is: 

•  0.771  if  vendor  1  is  selected 

•  0.800  if  vendor  2  is  selected 


•  This  new  metric  suggests  DoD  should  select 
vendor  2 
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Vendor  Selection  Metric  (VSM) 


Bid  Utilities 


The  buyer’s  utility  function  and  the  value  and  corresponding  utility  offered  by  each 
vendor  for  the  six  budget  scenarios  in  the  decision  under  uncertainty  example. 
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Interpretation  of  Results: 

•  Dividing  new  vendor  selection  metric  (VSM)  into 
component  parts  provides  additional  insight. 

•  Even  though  Vendor  2  wins,  the  VSM  values  for 
each  vendor  are  fairly  close: 

•  Vendor  1  =  0.771,  and  Vendor  2  =  0.800 

-  However,  for  budget  levels  $15,  $20,  $25,  and  $30, 
the  bundle  of  attributes  provided  by  vendor  1  is 
more  desirable,  and  there  is  a  75%  probability  one 
of  these  budget  levels  will  be  realized! 

-  But  there  is  a  10%  probability  of  a  serious  budget 
cut  to  $5  in  which  case  vendor  2  provides  a 
substantially  superior  offer. 


•  Such  insights  would  be  impossible  to 
obtain  with  only  a  single  bid  from  each 
vendor,  say  for  the  most  likely  budget,  b  = 
$15,  with  a  probability  of  0.35. 


•  More  revealing  and  robust  analysis  is  only 
feasible  if  DoD  solicits  vendor  offers  over 
multiple  budget  levels  and  assesses  the 
likelihood  of  those  budgets. 
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Recommendations 


•  Allow  vendors  to  submit  bids  for  a  range 
of  possible  funding  levels 

-  Full  Funding=Optimistic;  Partial  Funding=Most  Likely;  Limited 
Funding=Pessimistic. 


•  Instead  of  viewing  each  vendor  as  a  single 
point  in  cost-effectiveness  space,  it  is 
important  to  solicit  vendor  offers  at 
different  levels  of  affordability. 
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Recommendations 

•  A  vendor  whose  bid  is  dominated  at  one  budget 
level  could  be  the  winner  at  another  budget 
level. 

•  This  makes  it  vital  for  procurement  agencies  to 
rethink  traditional  public  sector  bid  solicitations. 

•  Develop  expansion  paths  to  illustrate  how  each 
vendor’s  offer  changes  with  changes  in  funding. 
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Recommendations 


•  With  increased  budget  uncertainty,  assign  a 
probability  distribution  over  possible  budgets 
(funding/affordability  levels). 

•  Develop  a  Vendor  Selection  Metric  (VSM) 

that  captures  budget  uncertainty  and  DoD’s 
attitude  towards  risk. 


Calculate  VSM  value  for  each  set  of  vendor 
proposals  and  use  to  guide  vendor  selection 

decisions.  STOP 
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□Suppose  the  buyer  has  the  exponential  expected 
utility  function  below  where,  as  previously  specified,  V 
varies  between  zero  and  one  over  the  possible  attribute 
bundles.  This  vendor  selection  metric  (VSM)  represents 
a  decision-maker  who  is  risk  averse. 


Note  that  since  the  minimum  value  of  V\s  zero  and  the  maximum  is  one,  U(V)  also 
varies  between  zero  and  one.  We  chose  the  exponential  function  because  it  has 
constant  absolute  risk  aversion,  measured  by  a  risk  tolerance  parameter  (in  this 
case,  0.5),  making  its  assessment  reasonably  straightforward.  It  is  commonly  used 
in  decisions  under  uncertainty. 


Examples 


•  Let  the  vendors  have  cost  functions  of  the 
form: 

cij ( aij)  =  ,  where  «(/ , Pv  >  0 


•  8-|=5,  82=10,  83=15,  84=20,  85=25,  86=30 


•  We  will  examine  several  cases  where  the 
vendors  differ  in  their  cost  functions  and/or 
beliefs  about  the  weight  the  buyer  places  on 
the  attributes 
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Solution  to  Vendor’s  Problem 


•  A  vendor’s  best  offer  (bid)  will  consist  of  the 
combinations  of  attribute  levels  that  use  the 
entire  possible  budgets,  and  satisfy  the 
condition: 


This  set  of  offers  from  a  vendor  constitutes  an 
“expansion  path” 
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